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Abstract 

Objective To examine whether the age based quality measure for 
screening for colorectal cancer is associated with overuse of screening 
in patients aged 70-75 in poor health and underuse in those aged over 
age 75 in good health. 

Design Retrospective cohort study utilizing electronic data from the 
Veterans Affairs (VA) Health Care System, the largest integrated 
healthcare system in the United States. 

Setting VA Health Care System. 

Participants Veterans aged >50 due for repeat average risk colorectal 
cancer screening at a primary care visit in fiscal year 2010. 

Main outcome measures Completion of colonoscopy, sigmoidoscopy, 
or fecal occult blood testing within 24 months of the 2010 visit. 

Results 399 067 veterans met inclusion/exclusion criteria (mean age 
67, 97% men). Of these, 38% had electronically documented screening 
within 24 months. In multivariable log binomial regression adjusted for 
Charlson comorbidity index, sex, and number of primary care visits, 
screening decreased markedly after the age of 75 (the age cut off used 
by the quality measure) (adjusted relative risk 0.35, 95% confidence 
interval 0.30 to 0.40). A veteran who was aged 75 and unhealthy (in 
whom life expectancy might be limited and screening more likely to result 
in net burden or harm) was significantly more likely to undergo screening 
than a veteran aged 76 and healthy (unadjusted relative risk 1 .64, 1 .36 
to 1.97). 

Conclusions Specification of a quality measure can have important 
implications for clinical care. Future quality measures should focus on 



individual risk/benefit to ensure that patients who are likely to benefit 
from a service receive it (regardless of age), and that those who are 
likely to incur harm are spared unnecessary and costly care. 

Introduction 

Quality measures are metrics used to assess the quality of 
healthcare. The definitions, or specifications, for quality 
measures are commonly derived directly from clinical practice 
guidelines. One example is the Healthcare Effectiveness Data 
and Information Set (HEDIS) measure, which seeks to decrease 
underuse of screening for colorectal cancer in the United States. 
The current HEDIS measure is based on the screening 
recommendations of the US Preventive Services Task Force 
(USPSTF) guidehne, which suggests routine screening in 
patients at average risk starting at the age of 50 and continuing 
through to the age of 75. ' The United Kingdom National Health 
Service (NHS) has proposed a similar age based measure for 
the bowel cancer screening program in the UK.^ Large integrated 
healthcare organizations in the US, such as the Veterans Affairs 
(VA) Health Care System, have also introduced age based 
quality measures as part of their accountability and improvement 
programs; indeed, VA has an active measurement and feedback 
system in place to encourage screening in age eligible 
individuals.' 

Despite the use of these well established age based quality 
measures, experts recommend that the decision to screen be 
informed by estimated life expectancy rather than age alone.'* ^ 
Age is a reliable proxy for life expectancy in young patients 
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(long life expectancy) and very elderly patients (short life 
expectancy). But in most elderly patients eligible for screening, 
life expectancy varies considerably according not only to age 
but also health status. For instance, a 74 year old man who is 
in excellent health has a life expectancy of almost 15 years. ^ 
Evidence suggests that screening for colorectal cancer is likely 
to be of benefit in such a patient and should therefore be 
encouraged.'' On the other hand, a 74 year old man who is in 
poor health has a life expectancy of less than five years.'' 
Evidence suggests that screening by any method is unlikely to 
be of benefit and might even be harmful in such a patient and 
should therefore be discouraged.'' ' In fact, the American College 
of Physicians (ACP) guideline for screening for colorectal cancer 
specifically recommends that screening should not be performed 
in individuals with a life expectancy of less than 10 years. Yet, 
as operationalized, the quality measure for screening for 
colorectal cancer encourages screening equally for both healthy 
and frail patients aged 74, potentially leading to overuse. 
Conversely, the specification of a maximum age beyond which 
performance of screening is not "counted" could promote 
underuse in healthy patients who are older than 75 but are still 
likely to benefit because of good life expectancy." ' As 
highlighted by Walter and colleagues in 2004, such an approach 
fails to sufficiently consider "who received the test, why it was 
performed, or whether the patient wanted it.""' 

We examined whether the upper age cut off of the quality 
measure for colorectal cancer screening is associated with 
overuse of screening among patients aged 70-75 in poor health 
and underuse in those aged over 75 in good health. We used 
data from the VA Health Care System, the largest integrated 
healthcare system in the US. Specifically, we compared 
screening between two groups of patients: older individuals 
who were within the target age range in the quality measure 
(aged 70-75) and older individuals who were beyond the age 
where screening "counts" toward the quality measure (age >75). 
We hypothesized that patients who were within the target age 
range but were in poor health (low likelihood of benefit from 
screening and potential for net harm) would be more likely to 
undergo screening than patients who were beyond the target 
age range but were in good health (higher likelihood of benefit 
from screening). We sought to examine how quality measures 
that utilize "one size fits all" targets without specific regard to 
individual risk and benefit influence both appropriate and 
inappropriate use of care. 

Methods 

VA Health Care System 

We performed a retrospective cohort study using electronic data 
from the VA Health Care System. This system serves nearly 
nine million veterans (that is, people who previously served in 
the US military) across over 150 medical centers and nearly 
1400 community based outpatient clinics. Though VA is under 
a single centralized leadership, its organizational structure allows 
for substantial regional and local autonomy in terms of 
budgetary decisions and approaches to delivery of care. The 
healthcare system has a robust centrally controlled quality 
measurement program, with a wide array of quality measures 
across various healthcare conditions.' In parallel, medical centers 
have adopted computerized reminders that prompt providers to 
fulfill quality measures for services such as screening for 
colorectal cancer. Audits of performance are also reported to 
providers at many medical centers, further encouraging 
screening. Partly as a result of such approaches, VA is 



considered a leader in the delivery of high quality care in the 
US." 

VA physicians are salaried, but a small percentage of their full 
compensation comes in the form of "performance pay," which 
can reward physicians for productivity, performance on quality 
metrics such as screening for colorectal cancer, or other locally 
determined factors. Many older veterans are eligible for 
Medicare insurance, which covers non-VA outpatient care. 
Thus, a considerable proportion of veterans receive care through 
both VA and private physicians.'' Although guidelines suggest 
that screening is not universally required in older patients, 
neither VA, Medicare, nor other private insurers currently limit 
coverage of screening for colorectal cancer in older adults in 
the US. 

Study design and dataset 

VA electronic data are available from fiscal year 2000 to the 
present time, including data on demographics, utilization of 
primary care and screening for colorectal cancer, and 
comorbidity status. We identified veterans who were regular 
users of VA primary care and preventive care and were due for 
repeat screening in the fiscal year 2010 (allowing for sufficient 
time to determine whether screening was subsequently utilized). 
Patients were included if they met the following criteria: primary 
care visit in 20 10 when the patient was due for repeat colorectal 
cancer screening (age >50, no fecal occult blood test (FOBT) 
in the 12 months before the 2010 visit, no sigmoidoscopy in the 
five years before the 2010 visit, and no colonoscopy in the 10 
years before the 2010 visit); and at least one additional primary 
care visit in the 12-24 months before the 2010 visit (to focus 
on patients who regularly use VA primary care). A similar 
approach has been used in other studies to identify regular users 
of VA primary care.'' 

To identify individuals who underwent screening for colorectal 
cancer through VA, we chose to focus on individuals who had 
undergone a previous fecal occult blood test with negative 
results in the 12-24 months before the 2010 visit. We made this 
decision on the basis of pilot work at a single VA medical center 
where we had access to more detailed clinical data, which 
showed that many veterans who seemed to be due for screening 
based on VA electronic data had in fact undergone previous 
colonoscopy screening outside VA (that is, VA electronic data 
are insensitive for detecting historical exposure to colonoscopy). 
Inclusion of these patients who were not due for screening in 
our cohort would lead to substantial underestimation of 
screening utilization. We carried out a sensitivity analysis to 
examine the impact of this inclusion criterion related to fecal 
occult blood testing on our results. 

Fecal occult blood tests were identified by using Logical 
Observation Identifiers Names and Codes (LOINC), and 
sigmoidoscopies and colonoscopies were identified by using 
ICD-9 (international classification of disease, ninth revision). 
Current Procedural Terminology (CPT), and Level II Healthcare 
Common Procedure Coding System (HCPCS) codes (appendix 
table A). Patients were excluded if CPT or ICD-9 codes showed 
any of the following diagnoses between 2000 and the qualifying 
2010 visit (appendix table B): previous colectomy; history of 
colon polyps; history of colorectal cancer; history of 
inflammatory bowel disease; or family history of colorectal 
cancer. These exclusion criteria were selected to ensure that the 
cohort comprised individuals who were at average (rather than 
increased) risk for colorectal cancer. 
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Outcome and predictor variables 

We determined whether any screening test (fecal occult blood 
test, sigmoidoscopy, or colonoscopy) was completed within 24 
months after the qualifying 2010 visit (primary outcome 
variable). A 24 month follow-up window has been used in 
previous work on this topic as it allows patients ample time to 
schedule and complete a screening test.'" Secondary outcome 
variables included fecal occult blood test, sigmoidoscopy, or 
colonoscopy performed within 24 months specifically for 
screening (that is, tests that were potentially performed for 
diagnostic purposes were excluded); fecal occult blood test 
within 24 months; sigmoidoscopy within 24 months; and 
colonoscopy within 24 months. To identify screening 
sigmoidoscopies and colonoscopies, we utilized an algorithm 
previously developed by Fisher and colleagues, using VA 
administrative data.'^ To identify screening fecal occult blood 
tests, we excluded patients in whom ICD-9 codes showed a 
diagnosis of anemia or bleeding in the three months before the 
fecal occult blood test (indicating that it might have been 
performed for diagnostic purposes) (appendix table B). 

Data were also extracted on several predictor variables, including 
demographic characteristics (age and sex), number of primary 
care visits in 2010, and health status. Race was not utilized as 
a covariate as it has been shown to have limited reliability in 
VA data." 

Health status was measured with the Deyo adaptation of the 
Charlson comorbidity index," based on inpatient and outpatient 
codes from the 24 months before and including the qualifying 
visit in 2010. The Charlson comorbidity index is a weighted 
score of 17 common comorbid conditions. Originally developed 
as a predictor of one year mortality after hospital discharge,'" 
subsequent work has shown that it can also be used as a predictor 
of long term mortality." In our study, the index was categorized 
according to the approach used by Walter and colleagues, in 
studies of screening in veterans (Charlson index 0, 1-3, or >4)J '* 
In a 75 year old man, a Charlson index of 0 indicates good 
health, with an estimated life expectancy over 10 years. ^ " An 
individual with a Charlson index of 0 might have mild comorbid 
conditions, such as uncomplicated hypertension or osteoarthritis, 
which do not substantially affect life expectancy. Such patients 
are likely to benefit from screening for colorectal cancer.** 
On the other extreme, a 75 year old man with a Charlson index 
>4 has poor health, with an estimated life expectancy of less 
than five years.' A patient with a Charlson index >4 is likely 
to have multiple comorbid conditions that impact life 
expectancy, such as diabetes with end organ damage, congestive 
heart failure, and emphysema. Such patients are unlikely to 
benefit from (and might even be harmed by) screening.'* '° 

Statistical analysis 

Screening completion rates were stratified by age at the time of 
the qualifying 2010 visit (50-69, 70-75, or >75), sex, Charlson 
comorbidity index, and number of primary care visits in 2010. 
We grouped 70-75 year olds together because previous research 
suggests that for individuals in this age range with a high burden 
of comorbidity life expectancy is similar and is less than the 10 
years needed for benefit from screening.''"'' We carried out 
multivariable log binomial regression to identify independent 
predictors of screening utilization within 24 months of the 
qualifying 2010 visit and to calculate adjusted screening rates. 
Robust standard errors were used to adjust for clustering within 
facility. We also performed sensitivity analyses using our 
secondary outcome variables and broadening our inclusion 
criteria to include patients both with and without previous fecal 



occult blood testing. Analyses were performed with the Stata 
12 statistical package (StataCorp, College Station, TX). The 
proportion of patients with missing data was negligible (date 
of birth was missing for 400 out of 4 756 477 patients). These 
400 patients were excluded (appendix fig A). 

Results 

We identified 399 067 veterans who met our inclusion and 
exclusion criteria (table Hi). The mean age was 67.2 (SD 9.7) 
years, and most were men. The median Charlson comorbidity 
index was 1 (interquartile range 1-3). Participants made a median 
of three visits to primary care clinics in fiscal year 2010. 

About 38% (151 850/399 067) of identified veterans (age >50 
with no upper age cut off) underwent a screening test for 
colorectal cancer (fecal occult blood test, sigmoidoscopy, or 
colonoscopy) in the 24 months after their qualifying 2010 
primary care clinic visit (table IJJ.)- Screening was more common 
among younger and healthier veterans (and among those with 
a greater number of visits to a primary care clinic) (table 2Ji). 
Screening was relatively stable from age 50 to age 75, but 
utilization declined abruptly after 75, matching the age cut off 
promoted by the existing VA quality measure (table 2 and figs 
l-2||i,l). This finding was independent of comorbidity status, 
number of primary care visits, and demographic characteristics 
(fig 311). Of 53 346 veterans aged 70-75, 16.5% (8814/53 346) 
had a Charlson comorbidity index >4 (indicating poor health 
and shortened life expectancy). Though these veterans were 
unlikely to benefit from screening, 40% (35 16/88 14) underwent 
a screening test within 24 months. On the other hand, of 97 786 
veterans aged >75, 24.6% (24 088/97 786) had a Charlson 
comorbidity of 0 (indicating excellent health and good life 
expectancy). Though many of these veterans would likely 
benefit from screening, only 16.5% (3964/24 088) underwent 
a screening test within 24 months. In fact, a veteran who was 
aged 75 and unhealthy (Charlson index >4) was significantly 
more likely to undergo screening than a veteran who was aged 
76 and healthy (Charlson index 0) (unadjusted relative risk 1 .64, 
95% confidence interval 1.36 to 1.97; fig 211). 

We performed several additional analyses to determine whether 
our inclusion and exclusion criteria (which were designed to 
identify regular users of VA primary care who seemed to be 
receiving colorectal cancer screening in VA) and primary 
outcome variable (any fecal occult blood test, sigmoidoscopy, 
or colonoscopy within 24 months of the qualifying 2010 visit) 
might have resulted in spurious associations between 
characteristics of the quality measure and screening utilization. 
First, we repeated our analysis for all veterans with regular use 
of VA primary care irrespective of prior fecal occult blood 
testing. Second, we repeated our analysis using secondary 
outcome variables, including any fecal occult blood testing 
within 24 months of the 2010 visit; any sigmoidoscopy within 
24 months; any colonoscopy within 24 months; and fecal occult 
blood testing, sigmoidoscopy, or colonoscopy within 24 months 
for screening purposes only (that is, diagnostic fecal occult 
blood tests, sigmoidoscopies, and colonoscopies were excluded). 
These sensitivity analyses showed similar results to our primary 
analysis, with an abrupt decrease in screening in veterans after 
age 75 independent of Charlson comorbidity index and number 
of primary care visits (appendix tables C and D). 
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Discussion 

Principal findings and comparison witli other 
studies 

Quality improvement efforts that incorporate quality measures 
have been shown to be effective in promoting utilization of 
appropriate care." But as our work and that of others has shown, 
quality measures also have the potential to encourage utilization 
of inappropriate care. This is particularly true of measures 
that use a "one size fits all" or "treat to target" approach that is 
designed to encourage adherence to clinical practice guidelines 
(like the measure for screening for colorectal cancer).'" As our 
study suggests, such an approach can unintentionally discourage 
thoughtful clinical decision making. Fisher and colleagues found 
similar results in a 2005 study of veterans, reporting that quality 
improvement efforts to increase screening for colorectal cancer 
were accompanied by high rates of inappropriate use of 
screening fecal occult blood tests."' Work in other clinical 
domains also supports our results. For example, Kerr and 
colleagues recently reported that high facility level performance 
on "treat to target" quality measures for veterans with diabetes 
was associated with overtreatment of both hypertension and 
hyperlipidemia. " Efforts such as the Too Much Medicine and 
Choosing Wisely campaigns have sought to limit overuse by 
encouraging patients and providers to discuss the 
appropriateness of care when the value of a service might be 
low.'*" But a patient and provider cannot have a meaningful 
discussion about whether to utilize a clinical service if they do 
not fully understand the value of care for that individual patient, 
nor if the provider is incentivized to order the service regardless 
of a patient's characteristics or preferences. Thus, refining 
quality measures to be more individualized and sensitive to 
preferences is critical if we want to promote care that is both 
more appropriate and patient centered."^ A patient centered 
approach to quality measurement, one that incorporates health 
status and individual preferences, is particularly needed in older 
patients in whom age alone is an inadequate proxy for life 
expectancy (such as those aged 70-80). This need will only grow 
as the population ages (yielding increasing numbers of healthy 
and unhealthy older patients) and as quality improvement 
initiatives that incorporate quality measures are more widely 
adopted. Recent work by Cho and colleagues has shown that 
use of administrative data to estimate life expectancy can be 
further tailored not only by health status but also by race and 
sex.^' Such an approach could be utilized by healthcare systems 
to better target screening to those individuals who are more 
likely to benefit. 

Strengths and limitations 

Our study has several strengths and limitations. First and 
foremost, we utilized data from the largest integrated healthcare 
system in the US and analyzed data on nearly 400 000 patients. 
Furthermore, the use of VA data allowed us to examine the 
impact of a longstanding, comprehensive, and highly effective 
implementation of quality measures on both underuse and 
overuse. Because performance of screening for colorectal cancer 
among patients aged 50-75 is now a standard measure in many 
healthcare systems, it is likely that similar patterns of underuse 
and overuse can be found in other high performing healthcare 
systems.^* The use of VA data also presents some limitations. 
Specifically, because veterans might undergo procedures outside 
VA, our data probably underestimate true screening rates. 
Indeed, previous work that utilized abstraction of medical 
records to assess receipt of screening for colorectal cancer has 
shown that about 80% of VA patients aged 50-75 are up to date 



for screening.^' We sought to resolve this limitation in our 
primary analysis by including only veterans who had undergone 
previous fecal occult blood test screening in VA, but even these 
veterans might undergo subsequent screening outside VA. 
Nonetheless, it is unlikely that veterans aged 75 differ 
substantially from those who are 76 in terms of use of screening 
outside VA (fig 20). It is also important to note that our study 
is retrospective in design, and the observed results could, at 
least partly, be caused by unmeasured confounding. For instance, 
we could not adjust for some variables that are known to impact 
screening utilization, such as socioeconomic status, race, and 
education, though these factors are also not likely to vary 
substantially among veterans aged 75 versus 76. Additionally, 
electronic data are subject to imprecision in coding, which could 
result in both overestimation or underestimation of screening 
utilization. 

Conclusions and policy implications 

Our study suggests that the way quality measures are specified 
can have a profound effect on screening utilization. In the case 
of colorectal cancer screening, the translation of the US 
Preventive Services Task Force guideline into a quality measure 
has resulted in high adherence to guideline recommendations. 
But screening does not seem to be utilized according to expected 
benefit across the population. Specifically, screening is likely 
being overused in sick patients aged 75 and younger and 
underused in healthy patients aged over 75. Of note, both the 
task force guideline and the VA Clinical Preventive Services 
Guidance Statement encourage an individualized approach to 
screening for colorectal cancer in patients aged 76-85, but this 
nuance had not explicitly been incorporated into the quality 
measure. In the past year, however, VA has modified the quality 
measure specifications to permit exceptions for veterans age 
<75 with shortened life expectancies or serious comorbid 
medical conditions. Though this is an important first step 
towards a more individualized approach, future patient centered 
quality measures will need to incorporate more explicit measures 
of clinical benefit and ultimately help guide clinical decision 
making. In this way, we can ensure that patients who are likely 
to benefit from a service receive it (regardless of age) and those 
who might incur more harm than benefit are spared unnecessary 
and costly tests and treatments. 
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What is already known on this topic 

Screening for colorectal cancer is an evidence based widely recommended preventive service that has traditionally been underused 

Efforts to increase screening through quality measurement have been successful, and such efforts are now being implemented widely 
in the US and the UK 

The implementation of "one size fits all" age based measures might have unintended effects on screening 
What this study adds 

In the largest integrated healthcare system in the US, age based measures for screening for colorectal cancer (which encourage 
screening in patients aged 50-75) promoted overuse of screening in unhealthy older patients and underuse in healthy older patients 

Future quality measures should focus on clinical benefit rather than simply chronological age to ensure that patients who are likely to 
benefit from a service receive it (regardless of age), and that those who are likely to incur harm are spared unnecessary and costly care 
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Tables 



Table I Baseline characteristics in study of role of quality measurement in inappropriate use of screening for colorectal cancer. Figures 
are numbers (percentage) of patients (n=399 067) 



Characteristic 


Participants 


Age 50-69 


Age 70-75 Age >75 


Age (years): 


50-69 


247 900 (62) 


247 900 




70-75 


53 351 (13) 




53 351 — 


>75 


97 816 (25) 




— 97 816 


Men 


385 781 (97) 


236 832 (96) 


52 499(98) 96 450(99) 


Women 


13 286 (3) 


1 1 068 (4) 


852 (2) 1366(1) 


Charlson comorbidity index: 


0 


127 968 (32) 


89 259 (36) 


14 621 (27) 24 088 (25) 


1-3 


213 814 (54) 


129 210(52) 29 916(56) 54 688(56) 


>4 


57 285 (14) 


29 431 (12) 


8814 (17) 19 040(19) 


Primary care visits in 2010: 


1 


49 191 (12) 


28 086 (11) 


6968 (13) 14 137(14) 


2 


88 099 (22) 


51 686 (21) 


12 561 (24) 23 852 (24) 


3 


75 115 (19) 


46 694(19) 


10 351 (19) 18 070 (18) 


>4 


186 662 (47) 


121 434(49) 23 471 (44) 41 757(43) 


Screening utilization*: 


Colonoscopy, sigmoidoscopy, or FOBT 


151 850 (38) 


113 835 (46) 


22 757 (43) 15 258(16) 


Colonoscopy 


37 237 (9) 


31 430 (13) 


3549 (7) 2258 (2) 


Sigmoidoscopy 


2008 (1) 


1614 (1) 


201 (0.5) 193 (0.2) 


FOBT 


129 103 (32) 


94 725 (38) 


20 806 (39) 13 572(14) 


FOBT=fecal occult blood test. 
'Categories are not mutually exclusive. 
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Table 2| Unadjusted and adjusted* relative risksf and rates of screening for colorectal cancer by age and health status (n=399 067) 


Unadjusted 




Adjusted 


Predictor variable RR (95% CI) 


Screening rate, % (95% CI) 


RR (95% CI) 


Screening rate, % (95% CI) 


Age (years) : 


50-69 1 (reference) 


45.9 (45.7 to 46.1) 


1 (reference) 


45.6 (43.7 to 47.4) 


70-75 0.93 (0.91 to 0.95) 


42.7 (42.2 to 43.1) 


0.94 (0.92 to 0.96) 


42.9 (40.9 to 44.9) 


>75 0.34 (0.30 to 0.39) 


15.6 (15.4 to 15.8) 


0.35 (0.30 to 0.40) 


15.8 (13.6 to 18.1) 


Ctiarlson comorbidity index^:: 


0 1 (reference) 


41.0 (40.7 to 41 .3) 


1 (reference) 


38.7 (36.8 to 40.5) 


1-3 0.92 (0.90 to 0.93) 


37.5 (37.3 to 37.7) 


0.94 (0.93 to 0.95) 


36.0 (34.2 to 37.7) 


>4 0.82 (0.80 to 0.83) 


33.4 (33.0 to 33.8) 


0.87 (0.85 to 0.89) 


33.1 (31 .6 to 34.6) 



'Adjusted for sex and number of primary care visits in 201 0; unadjusted results reflect simple proportions witfiin each age or comorbidity group. 
tP<0.001 for all comparisons. 

ijiln a 75 year old man, Charlson index of 0 indicates life expectancy >10 years, 1-3 indicates life expectancy 5-10 years, and >4 indicates life expectancy <5 years. 
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Figures 
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Fig 1 Relation between age, health status, and screening (n=399 067). In a 75 year old man, Charlson index of 0 indicates 
life expectancy >10 years, Charlson index of 1-3 indicates life expectancy of 5-10 years, and Charlson index >4 indicates 
life expectancy <5 years 
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Fig 2 Screening at age 75 i^age 76 (n=21 499) 
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Fig 3 Screening by age (n=399 067) adjusted for sex, Charlson comorbidity index, and number of primary care visits in 
2010 
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